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Abstract 

Background: Despite the critical role of nursing care in determining high-performing healthcare delivery, 
performance science in this area is still at an early stage of development and nursing's contribution most often 
remains invisible to policy-makers and managers. The objectives of this study were: 1) to develop a theoretically 
based framework to conceptualize nursing care performance; 2) to analyze how the different components of the 
framework have been operationalized in the literature; and 3) to develop a pool of indicators sensitive to various 
aspects of nursing care that can be used as a basis for designing a performance measurement system. 

Methods: We carried out a systematic review of published literature across three databases (MEDLINE, EMBASE and 
CINAHL), focusing on literature between 1990 and 2008. Screening of 2,103 papers resulted in final selection of 101 
papers. A detailed template was used to extract the data. For the analysis, we used the method of interpretive 
synthesis, focusing first on 31 papers with theoretical or conceptual frameworks; the remaining 70 articles were 
used to strengthen and consolidate the findings. 

Results: Current conceptualizations of nursing care performance mostly reflect a system perspective that builds on 
system theory, Donabedian's earlier works on healthcare organization, and Parsons' theory of social action. Drawing 
on these foundational works and the evidence collated, the Nursing Care Performance Framework (NCPF) we 
developed conceptualizes nursing care performance as resulting from three nursing subsystems that operate 
together to achieve three key functions: (1) acquiring, deploying and maintaining nursing resources, (2) 
transforming nursing resources into nursing services, and (3) producing changes in patients' conditions. Based on 
the literature review, these three functions are operationalized through 14 dimensions that cover 51 variables. The 
NCPF not only specifies core aspects of nursing performance, it also provides decision-makers with a conceptual 
tool to serve as a common ground from which to define performance, devise a common and balanced set of 
performance indicators for a given sector of nursing care, and derive benchmarks for this sector. 

Conclusions: The NCPF provides a comprehensive, integrated and theoretically based model that allows 
performance evaluation of both the overall nursing system and its subsystems. Such an approach widens the view 
of nursing performance to embrace a multidimensional perspective that encompasses the diverse aspects of 
nursing care. 

Keywords: Performance measurement. Nursing care. Systems theory. Quality of care. Nursing structure. Nursing 
processes. Nursing sensitive outcomes. Systematic review. 
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Background 

Nurses represent the largest occupational group in the 
healthcare workforce, providing the most care at all 
levels of the care continuum and accounting for a sig- 
nificant proportion of hospitals' operating costs [1,2]. 
Yet nursings contribution most often remains virtually 
invisible to policy-makers and healthcare managers, and 
many analysts consider it undervalued and understudied 
[3,4]. Symptomatic of the situation, data considered sen- 
sitive to nursing care are, in most cases, not represented 
in databases that are routinely examined for perform- 
ance analysis in healthcare organizations and health pol- 
icy decisions [5,6]. The causes attributed to these 
symptoms include poor conceptualization of nursing 
performance, inadequate measures of nursing contribu- 
tion, inadequate information systems to capture and ma- 
nipulate nursing performance data, and absence of a 
standardized language [7,8]. 

Concerns about the shortage of nurses and its poten- 
tial adverse effect on patient safety, coupled with mount- 
ing public expectations regarding the value of services 
consumed, have created a sense of urgency about the 
need for monitoring performance of nursing services [9]. 
In the current context of health service reforms, driven 
partly by resource constraints and consumer pressures, 
increasing demands are being placed on nursing admin- 
istrators and nursing care providers. They must ensure 
the performance of nursing services, give a more com- 
prehensive and accurate picture of what they do, and 
demonstrate the value and benefits of their services in 
line with established objectives and standards [10,11]. 
This renewed interest has resulted in an accelerated ex- 
pansion of a range of initiatives within and outside the 
nursing profession to make explicit those aspects of care 
outcomes directly attributable to nursing practice. Spe- 
cifically, indicators to quantify nursing s contribution to 
healthcare performance have been considered in several 
countries by influential organizations such as the Agency 
for Healthcare Research and Quality and the National 
Quality Forum in the United States, the Ontario Hos- 
pital Association in Canada, and the Council on Health 
Care Standards in Australia [12,13]. 

Despite such recent efforts, the research base on per- 
formance measurement in the area of nursing care is 
still considered somewhat embryonic [14], and no com- 
mon theory-driven schema guides the nursing discipline, 
the regulatory agencies and the provider organizations 
in their efforts to define, organize and operationalize the 
dimensions of nursing care performance [15]. There are 
many fragmented pieces of knowledge, but what is crit- 
ically missing is a comprehensive framework in which 
to fit the pieces together and that can guide the imple- 
mentation of performance assessment activities in nurs- 
ing. In many cases, the terminology of performance 



measurement can itself be confusing and is character- 
ized by a wide array of terms and concepts that include 
productivity, outcomes, effectiveness, efficiency, quality, 
etc. As there is no agreed-upon definition, performance 
measurement systems are also conceptualized in mul- 
tiple ways [16]. 

The overall aim of this research was to draw on the 
most recent developments in conceptual modelling to 
help build a common orientation on how to capture 
nursing care performance. More specifically, the objec- 
tives of the research were: 1) to develop a theoretically 
based framework to conceptualize nursing care perform- 
ance; 2) to analyze how the different components of the 
framework have been operationalized in the literature; 
and 3) to develop a pool of indicators sensitive to vari- 
ous aspects of nursing care that can be used as a basis 
for designing a performance measurement system. 

While there is already a substantial amount of litera- 
ture on the topic of healthcare performance, the focus of 
our study was on performance metrics potentially sensi- 
tive to nursing care. Although other reviews are available 
in the literature [17,18], none have been as exhaustive 
and none have systematically analyzed the existing 
models using a structured template, as we did. Further- 
more, while the notions of quality and performance are 
frequently used interchangeably, our study acknowledges 
both their conceptual linkages and differences. A system 
can only be said to be performing if it delivers high qual- 
ity interventions, care or services [19]. However, per- 
formance and quality are not necessarily identical and 
interchangeable concepts. A widely accepted definition 
of quality proposed by the Institute of Medicine concep- 
tualizes quality as the degree to which health services for 
individuals and populations are consistent with current 
professional knowledge and increase the likelihood of 
desired health outcomes [20]. From this perspective, 
quality is at best only a proxy of healthcare performance, 
which is understood as a much broader concept. To be 
consistent with developments in contemporary litera- 
ture, the notion of performance in this study is embed- 
ded in a general vision of key functions that an 
organizational entity must fulfill in a given environment. 
It refers to the ability of a health unit, organization, or 
system to perform its diverse functions most effectively 
and efficiently and to ensure the coordination and equi- 
librium among these functions needed to achieve its 
goals. From this perspective, nursing care performance 
can be measured based on those attributes or dimen- 
sions related to the functioning of organizational entities 
involved in nursing care provision. Those dimensions 
are not necessarily direct measures of quality, but rather, 
they cover what an organization does to provide nursing 
care, how it does it in relation to articulated goals and 
functions, and the resulting outcomes. 
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This article presents the Nursing Care Performance 
Framework (NCPF), describes its key components and 
identifies indicators related to each component. This 
NCPF will provide policy-makers and managers with an 
effective tool to structure performance evaluation sys- 
tems that are consistent with their overall goals. This 
framework will widen the view of nursing performance 
to encompass the diverse aspects of nursing care. 

Methods 

To achieve the above-mentioned objectives, our research 
built upon developments in conceptual modelling and 
advances in systematic review methods as applied to 
organizational research [21]. Our approach drew on ex- 
plicit, systematic and reproducible methodology to lo- 
cate, select and appraise the available literature and to 
extract and analyze the relevant data [22,23]. It can be 
defined as an interpretive synthesis. It is interpretive in 
the sense that the focus was not on simply aggregating 
or summarizing data reported in the literature but rather 
drawing on a large and complex body of evidence to 
build a conceptual framework grounded in the studies 
included in the review [24,25]. As summarized in 



Figure 1, the review followed the main phases that have 
been systematized in previous works: literature searches, 
screenings and appraisals, data extraction, and analysis 
and interpretative synthesis [21,22]. 

Literature search 

The search for relevant literature was made across three 
databases. MEDLINE, EMBASE and CINAHL were se- 
lected, based on preliminary tests applied to a broader 
range of databases. Other potential sources, such as the 
Cochrane and Francis Costello libraries, were rejected 
after preliminary tests found them unsatisfactory, as they 
generated hundreds of irrelevant items. After several it- 
erations and advice from a professional librarian, we 
adopted key words and applied them to the three se- 
lected databases. The key words were nursing care 
performance, nursing performance, nursing-sensitive 
performance measurement, nursing-sensitive measures, 
and quality indicators and nursing care. The robustness 
of the strategy was confirmed by comparison with a 
predetermined list of key articles referenced in a recent 
review article [17], all of which were captured by the 
search when they were clearly related to performance in 



Literature searches: Three scientific databases (IVIEDLINE, EMBASE and CINAHL) + 
Selected websites - Literature between 1990 and 2008 -^Collection of 2103 papers 



Screening by abstract by 3 pairs of 
reviewers after refinements to the inclusion 
criteria -^181 papers 



Download of the 181 papers and data 
extraction 

• Selection of 101 papers (including 31 
papers with explicit conceptual models) 
for the final phase of analysis 

• Extraction of basic information from the 
selected 101 papers by 3 pairs of 
reviewers 



Analysis and interpretative synthesis 




"Snowballing" 




> 





Development of a Nursing Care Performance Framework 



Figure 1 Methodological steps. 
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the area of nursing care. After removal of duplicates, 
the papers collated at this phase totalled 2,103. The 
search was limited to literature between 1990 and 2008, 
to focus on most recent developments in this area. 
However, reference chaining was used to uncover add- 
itional key articles published before this period. 

Screening and appraisal 

Our six researchers were paired into three teams and the ar- 
ticles' abstracts split among the teams. Inclusion criteria 
were tested on a sample of 15 abstracts. Each abstract was 
systematically reviewed independently by two researchers 
based on criteria of: 1) relevance (clear focus on the theme 
of performance; clear focus on the domain of nursing care); 
2) potential contribution to the conceptualization or meas- 
urement of performance (development of a theoretical 
model, elaboration of performance measures, performance 
evaluation, empirical analysis); and 3) use of explicit and 
valid methods. For each case, a decision was taken to either 
exclude the paper or select it for the next step. When there 
was disagreement, the two researchers met to discuss their 
views and find common ground. Letters, comments, edito- 
rials and anecdotal papers were systematically excluded. The 
criterion on methodological quality was intentionally broad 
for this initial screening. At this step, we aimed to include as 
many relevant papers as possible that met basic methodo- 
logical standards or were likely to contribute at the level of 
concepts, while excluding studies that were irrelevant or fa- 
tally flawed. One hundred and eighty-one papers passed this 
screening and advanced to the next step. 

Final screening and data extraction 

The 181 papers were downloaded. We designed a data ex- 
traction template that was intended, first, to confirm the 
relevance of each paper (clear focus on performance 
conceptualization and/or measurement in the domain of 
nursing care) and its scientific quality (theoretical founda- 
tion, design, sampling, data collection, explicit criteria of val- 
idity) and then to extract fiarther data. Again, the six 
researchers were grouped into teams of two. We pretested 
the data extraction tool on 45 articles split among the three 
teams, refined the tool, and then applied it to the entire 
sampling. In-depth examination of each article and data ex- 
traction were first completed by one member of a team and 
then validated by the other. It was still possible at this step 
to exclude a paper if it was deemed irrelevant or methodo- 
logically flawed. Again, when there was disagreement, the 
two researchers met to discuss their views and find common 
ground. Of the 181 articles, only 101 merited complete data 
extraction and analysis. 

Information extracted from all the 101 papers included 
country, type of study, appropriate criteria of validity 
based on the type of study (ex: representative sampling, 
data saturation), type of setting, study focus (development 



of framework, development of indicators, empirical ana- 
lysis of links between variables) and a list of performance 
indicators (variables or markers proposed or used to 
measure performance). The main context of interest was 
acute care for 65% of the papers, long-term care for 27% 
and primary care for 8%. This information was 
supplemented with grey literature, reference chaining and 
scanning of key journals specializing in performance is- 
sues. The three pairs of reviewers were also asked to flag 
papers with theoretical or conceptual frameworks; 31 such 
papers were identified on the basis of two criteria: 1) pa- 
pers with an explicit framework that specified the dimen- 
sions of the concept of nursing care performance and/or 
how these dimensions can be operationalized; and 2) re- 
views that used a coherently organized structure to 
present the various aspects of nursing care performance. 

Data analysis 

Our approach to analyzing the data involved constructing 
a general interpretation grounded in the findings of separ- 
ate studies and then integrating evidence from across 
studies into a coherent theoretical framework comprising 
a network of constructs [25,26]. For this phase, the ana- 
lysis focused first on the 31 papers with theoretical frame- 
works. Our approach was similar to that often undertaken 
in primary qualitative research. We began with detailed 
inspection of the papers, gradually identifying recurring 
variables used to measure performance. We then grouped 
these variables into different subdimensions and dimen- 
sions, constantly comparing the conceptual structures we 
were developing against the data in the papers and the 
models collated, and attempting to specify the relation- 
ships between the different variables, dimensions or 
subdimensions identified. The information extracted from 
the 70 other papers were incorporated in a second stage 
to strengthen and consolidate the findings drawn from the 
first 31 papers, confirm the performance indicators identi- 
fied and their groupings into dimensions and 
subdimensions, look for any additional variable or dimen- 
sion that would not have been captured in the models, 
and document the relationships between the variables 
identified. All team members were involved in this ana- 
lysis and the continual dialogue between them helped to 
ensure a reflexive account of the processes followed all 
along the different steps. 

Results and discussion 

A diversity of perspectives with a common thread 

A first observation that emerged from the corpus of pa- 
pers examined was that there are a significant number 
of activities under way to standardize measures of di- 
verse aspects of nursing work and capture nursing s con- 
tribution to care. The review revealed three paraUel 
streams of activities that have contributed to varying 
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degrees to conceptualizing and measuring performance 
in nursing care. 

A first stream of activities focuses on encoding and 
classif)^ing all aspects of nursing activities, including 
nursing diagnoses, nursing interventions and nursing 
outcomes [27,28]. Those classification systems are 
viewed as measurable and standardized ways to sort out 
those health conditions or clinical situations that are 
amenable to nursing interventions and for which nursing 
holds the ultimate responsibility for patient outcomes. A 
second stream of activities, grounded in the patient 
safety movement, is dominated by a search for evidence 
relating given factors in nursing structures or processes 
(e.g. nurse staffing, nursing unit characteristics) to safety 
failures or poor outcomes for patients [29,30]. A third 
stream of activities, closely associated with the quality 
assurance movement, is primarily concerned with devel- 
oping evidence-based standards of care and measuring 
resources, processes and interventions that must be in 
place to optimize care quality and outcomes [31]. 

While these activities follow different directions, the 
review also showed they share a common thread. In the 
three streams, nursing care performance involves the 
analysis of multiple interacting elements that relate to 
the diverse aspects of nursing services, their antecedents 
and their results. This reflects a conception of nursing 
care as a complex, aggregate entity, comprised of mul- 
tiple interrelated and interdependent subsystems and 
components that are logically coordinated and oriented 
toward the achievement of common goals [32]. As the 
review unfolded, a core finding that emerged from the 
analysis was that these diverse streams of activities, to 
varying extents, built upon Donabedians model of 
healthcare organization and delivery, which in turn owes 
much to Parsons' theory of social action and systems 
theory. Taken together, these theoretical foundations 
help us understand how inputs are acquired from the 
nursing care environment and fed into the service pro- 
duction cycle, where transformation of resources results 
in changes in patient conditions [33]. As such, they pro- 
vide a strong basis for organizing and integrating the di- 
verse and often fragmented perspectives of nursing care 
performance. 

Donabedians structure, processes and outcome (SPO) 
framework provides a tool that efficiently articulates 
three subdimensions representing three key components 
of the healthcare supply chain [34]. In many cases, the 
three components of the SPO model have been used as 
a taxonomy to identify and classify those factors or 
building blocks that must be put together to assess nurs- 
ing care performance. However, while the SPO triad 
highlights the different components and the underlying 
anatomy that define the performance of a given system, 
it may be not sufficient for understanding the operating 



mechanisms of these diverse components and their in- 
teractions. Parsons' framework for social analysis pro- 
vides a functional organismic perspective that affords 
further insight into the performance of a given system 
by conceptualizing the social system as a system of inter- 
actions between different subsystems, mostly defined in 
terms of functions [35]. These functions that any system 
needs to align and balance in order to perform well in- 
clude goal attainment, production, adaptation to the en- 
vironment, and values' maintenance. As such. Parsons' 
framework may be useful to understand the physiology 
of a performance system, conceptualized as a set of spe- 
cialized functions that must be achieved by autonomous 
functional subsystems. The Parsons functions provide a 
generic foundation that has been used to define the cri- 
teria on which human services performance must be 
assessed [36]. This is also congruent with systems the- 
ory. A systems approach to nursing care assumes that 
the nursing system is composed of interrelated subsys- 
tems and components that carry out specialized func- 
tions. If any of these subsystems performs inadequately, 
it will affect the performance of the whole system. This 
means that a highly complex nursing care system must 
first be broken into subsystems so that each can be ana- 
lyzed and understood before being reassembled into a 
whole. Each subsystem must achieve high performance 
in its own functions but also join with other subsystems 
and units to optimize the performance of the whole [37]. 
Although these functional subsystems may be relatively 
self-sufficient, a systems perspective recognizes the 
interaction and reciprocal interconnection among their 
different functions [37]. 

We used the complementarity of these foundational 
works to analyze the selected articles and determine the 
current state of performance measurement in nursing 
care. Drawing upon these works and the evidence col- 
lated, we developed the Nursing Care Performance 
Framework (NCPF) to integrate diverse perspectives and 
take into account the diverse aspects of the nursing sys- 
tem. Donabedian's SPO triad provides the underlying 
anatomy of this framework. Parsons' social theory helps 
to understand the operating mechanisms and functions 
that underlie each of the constitutive components. The 
system perspective takes into account the linkages be- 
tween those different components or subsystems and 
the environmental factors influencing them. The 31 
models we reviewed differ widely in the translation of 
these theoretical foundations and reveal diverse gaps in 
measuring nursing care performance. We observed wide 
differentials in the existing models and approaches re- 
garding how performance is conceptualized and subse- 
quently operationalized. Some models exhibit a plethora 
of indicators reflecting diverse theoretical, methodo- 
logical and practical perspectives. Other frameworks 
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focus on selected and restricted aspects of nursing prac- 
tice, while neglecting others. Structural indicators that 
may be more easily measurable and more readily avail- 
able in existing administrative data sets are often put 
forward to the detriment of more complex process indi- 
cators that could better reflect nursing activities. Several 
frameworks or models reviewed are simply a list of indi- 
cators or broad domains and do not clearly indicate the 
conceptual dimensions and subdimensions that underlie 
these indicators. While many models built on the 
Donabedian structure-process-outcomes (SPO) frame- 
work, often each component of the triad is taken separ- 
ately, without attention to the linkages between them. 
Variations among the existing models also reflect the 
lack of a common terminology and the use of a plethora 
of terms and concepts with inconsistent definitions. Des- 
pite these limitations, our analysis is built on the hypoth- 
esis that conceptualization of performance in the 
domain of nursing may benefit from the combination of 
different perspectives and models. Taken together and 
used in combination, guided by the theoretical founda- 
tions mentioned above, existing models may provide a 
more accurate picture of nursing system performance 
than would each one separately. 

The nursing care performance framework 

The NCPF, illustrated in Figure 2, conceptualizes nurs- 
ing care performance as resulting from three nursing 
subsystems that operate together to achieve three key 
functions: (1) acquiring, deploying and maintaining 
nursing resources, (2) transforming nursing resources 
into nursing services, and (3) producing positive changes 
in a patients condition as a result of providing nursing 
services. These three functions, derived initially from the 
theoretical foundations mentioned above, served as an 



analytical tool to capture the main dimensions emerging 
from efforts to conceptualize and measure performance 
in the articles reviewed. Table 1 shows how our pro- 
posed framework relates to Donabedians SPO triad. 
Parsons' functional analysis and systems theory and 
builds on these foundations to develop a broad view of 
nursing care performance. In the NCPF, each of the 
three subsystems is composed of multiple components 
that define the structure, processes and outcomes of the 
overall system. Within each subsystem, the components 
are involved in multiple processes of interaction to 
achieve a specialized function. The underlying systemic 
perspective highlights the linkages between these 
functions. 

Building upon the perspectives mentioned above, the 
NCPF suggests a broader view of nursing care perform- 
ance that can be defined as the capacity demonstrated 
by an organization or an organizational unit to acquire 
the needed nursing resources and use them in a sustain- 
able manner to produce nursing services that effectively 
improve patients' conditions. Contrary to many prevail- 
ing definitions, nursing care performance is not re- 
stricted to the end goals or outcomes of the nursing 
system, but refers also to the effectiveness of those up- 
stream functions that provide the means necessary to 
achieve nursing system goals. 

As shown in Tables 2, 3 and 4, the three functions of 
the NCPF summarize the universe of dimensions and 
subdimensions used to conceptualize nursing care per- 
formance in the 31 models collated (see the models list. 
Table 5). The same dimensions and subdimensions were 
found in the 70 additional articles (empirical analyses, 
descriptive articles) retained after the screening exercise. 
Tables 2, 3 and 4 show not only the extent to which 
the 31 performance models inventoried integrate the 



Nursing care environment: legislation, regulation, professional 
systems, social and economic issues, culture, educational systems 
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Figure 2 The nursing care performance frameworlc. 
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Table 1 Theoretical foundations for the NCPF (Nursing Care Performance Framework) 



NCPF three functions Donabedian's SPO Triad 



Parsons' functions 



Systems theory 



Acquiring, deploying 
and maintaining 
nursing resources 

Transforming nursing 
resources into nursing 
services 



Structure: refers to cliaracteristics 
tliat affect tine ability of the nursing 
system to meet healthcare needs 

Process: refers to the nature of 
activities done by nurses in 
providing care and the 
characteristics of the practice 
environment 



Producing changes in a Outcome: refers to states of health 

patient's condition as a or events that follow nursing care 

result of providing and are affected by nursing care 
nursing services 



Adaptation: relates to a nursing system's 
capacity to acquire and maintain the 
resources it needs, develop new resources 
or improve allocation of its resources 

Production: relates to the ability of the 
nursing system to coordinate the efforts of 
its constituencies and ensure smooth 
functioning of processes involved in 
providing nursing services 

Value maintenance: relates to creating 
and maintaining values and standards that 
guide choices in the design of nursing 
services 

Goal attainment: relates to a nursing 
system's capacity to fulfill its mission and 
bring about a valued state in the system's 
relationships to its environments 



Ability of the nursing system, as an 
open system composed of interrelated 
subsystems, to acquire inputs from its 
environment, engage in transformation 
processes and generate output that 
brings about an added value for its 
environment 



three subsystems, but also how each subsystem is 
operationalized in each model Assuming these models 
summarize the universe of content commonly accepted 
as the main components of nursing care performance, 
we listed all variables present in the 31 models. Variables 
with similar meanings but differences in denominations 
were renamed to facilitate their groupings under one 
heading. We present below the three nursing care sub- 
systems and their components. We also provide evi- 
dence on interrelations both between each subsystems 
components and between the subsystems themselves. 

Acquiring, deploying and maintaining nursing resources 

No system for healthcare delivery can fulfill its objective 
of providing care and improving patient health without 
deploying the necessary human and material resources. 
Therefore, as revealed by our review of the 31 models, 
the first key function of a nursing system is to acquire, 
deploy and maintain the resources needed to provide 
nursing care. Twenty-six of the models include indica- 
tors related to this function, which has been defined as 
the ability to ensure a sound and efficient stewardship in 
acquiring and managing the needed nursing resources 
[67]. To achieve this function, the nursing system must 
perform on four key dimensions (see Table 2). 

Supply of nursing staff 

Nursing care is labour-intensive, and delivering effective 
nursing services depends on the availability of staff with 
the skills and competencies necessary to address pa- 
tients' specific needs in a timely manner. The supply of 
nursing staff, as a performance measure, thus reflects 
the effectiveness of diverse activities that govern nursing 
staff intake (planning, recruitment, selection) and de- 
ployment and must ensure an adequate balance with the 
demand for nursing services. Supply is concerned with 



not only the quantity of available staff, but also their 
types (educational preparation, qualifications, and ex- 
perience) and their mix. It must also be examined in the 
capacity of available staff to address the specific needs of 
patients they care for, and this raises the importance of a 
resource allocation system that takes into account the 
acuity of patients' conditions. 

Nurses' working conditions 

Two aspects of this dimension emerged from the litera- 
ture reviewed. One was the diverse types of material re- 
sources that are required to support the work of nursing 
staff. These include physical facilities, technologies, 
organizational configurations and financial resources, all 
factors identified in several models as precursors that 
define the overall organization of nursing care and deter- 
mine the extent to which nursing staff are able to per- 
form their roles. The other aspect consisted of 
employment characteristics, which encompass a number 
of issues ranging from workload, scheduling and em- 
ployment status to system-wide issues such as labour re- 
lationships. This aspect reflects the ability to create 
conditions that may attract nursing workers and ensure 
their stability in the workforce. This aspect is captured 
in the inventoried models by a series of indicators, in- 
cluding balance between temporary and permanent staff, 
use of agency personnel, ratios of floating staff, balance 
between full-time and part-time staff, and ratios of over- 
time hours. 

Maintenance and economic sustainability of the nursing 
staff 

These two dimensions can be considered to a certain ex- 
tent as contingent upon activities developed in the two 
first dimensions (supply and working conditions). Poor 
working conditions and inadequate staffing, often 



Table 2 Operationalization of the subsystem of acquiring, deploying and maintaining nursing resources 



I I 
9- ^ 

o 5 



Acquiring, deploying and maintaining nursing 
resources 



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 # 

models 



1. Nursing staff supply 



Quantity/intensity 1 
Quality (training, experience) 
Sl<ill mix 

Patient classification systenns 



2. Working conditions 



a. Support resources 

Physicai faciiities 
Material resources 
b. Employment conditions 

Stability (Overtime/ agency nurses floating nurses) 

Workload 

3. Staff maintenance 

Satisfaction at work 
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Table 3 Operationalization of the subsystem of transforming nursing resources into relevant nursing services ^ 

Transforming nursing resources into relevant nursing 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 # 
services models 

1. Nursing processes 

Assessment planning & evaluation 1111111 11 9 

Problenns & synnptonns nnanagennent 11111 11 11 1111 13 

Pronnotion / Prevention 1 1 11111 7 

Hospital connnnunity integration/ Discharge planning 111 1 4 

Deployment of scope of practice 11 1 3 

2. Patient centrality in the nursing care delivery process 

Continuity (reactivity timeliness, coordination) 111 1 111 1 8 

Patient/family involvement (self-care/ information/ education) 1111 111 11 1111 13 

Responsiveness to patients' needs and expectations 111 1 4 

(communication, comprehensiveness) 

3. Nursing work environment 

Nursing work environment characteristics (perceived autonomy 1 1 1111 1 1 11 1 1 12 

role tension, collaboration) 

4. Professional satisfaction 1111 15 
Number of indicators 221325422327432302530251 53221 03 



0) 

vo 
o 

-h 
NJ 
O 
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Producing changes in patients' conditions 



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 # 

models 



1. Risk outcomes and safety 

Falls 1 
Injuries 

Medication management: errors and complications 1 
Pulmonary infections 
Pressure ulcers/skin integrity 
Urinary complications 
Intravenous infections 
Abuses 

Nosocomial infections (taken broadly) 1 
Failure to rescue 

2. Patient comfort and quality of life related to care 

Hygiene 

Physical and chemical restraints 

Symptoms management (e.g., pain, nausea, dyspnoea, 1 

fever) 

Incontinence 

Comfort and quality of life (taken broadly) 

3. Patient empowerment 

Ability to achieve appropriate self-care 
Adoption of health-promoting behaviours 
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Physical functional capacity 1 111111 1 111111 1 1 1 17 | ^ 

Cognitive and psychosocial functional capacity 11111111 111111 1 15 o" g 

Functional capacity (taken broadly) 111 3 S. > 
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Patient satisfaction/complaints 11111111 1 1 1 1 1111 1 17 •<] 



5. Patient satisfaction 



6. Joint contribution of nursing with other care 
systems 

Hospital mortality 11 1 1 11111 9 

Readnnissions 11 1 1 11 6 

Length of stay ] ] ] ] 1 1111 9 

Other connplications related to care interventions 11 1 11 117 

Number of outcomes indicators 6 5 10 10 4 9 7 12 2 12 1 13 11 8 5 8 8 6 3 8 6 10 10 7 15 5 12 3 9 7 6 
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Table 5 Models list 


# 

paper 


Model 


References 


1 


Conceptualization of nursing service organization as a system 


[10] 


2 


Nursing role effectiveness model 


[38] 


3 


Approaches for selecting and developing indicators 


[18] 


4 


Nursing role effectiveness model - Application to quality improvement 


[39] 


5 


Framework for quality of care with cell characteristics 


[40] 


0 


The health care vortex; acute care quality indicators,' community-based non-acute care indicators 


[4 IJ 


7 


Conceptual framework for evaluating the ACNP role (adapted from the Nursing Role Effectiveness Model) 


L4ZJ 


Q 

O 


SPO framework 




Q 


Process of nursing care ratings 


L44J 


1 n 
1 u 


SPO systems model of nursing care quality in nursing homes 




1 1 


Adapted framework from the Quality Health Outcomes Model (Mitchell et al., 1998) 


[46] 


1 1 
1 z 


Antecedents or structural criteria of quality nursing care - Process criteria of quality nursing care - Consequences or outcome 
criteria of quality nursing care 


M 71 
[4 /J 


13 


Patient safety related outcomes and adverse events per category of preventive nursing activity 


[48] 


14 


Conceptual framework for the RICH nursing study (Rationing of Nursing Care in Switzerland study) 


[49] 


15 


Expert panel concept model 


[50] 


16 


Ontario Hospital Report framework - Indicators in the nursing report 


[51,52] 


17 


Nursing Outcomes Classification 


[53] 


18 


A theory driven approach to evaluating quality of nursing care 


[54] 


19 


Nursing role effectiveness model 


[55] 


20 


Oncology Nursing Society Outcomes project team 


[56] 


21 


Nursing sensitive patient outcomes (NOC) 


[57] 


22 


Conceptual framework of ICU nursing workload 


[58] 


23 


The Spider diagram nursing quality report card 


[59] 


24 


ANA acute care nursing quality indicators 


[60] 


25 


Review - Summary of conceptual frameworks for measurement 


[17] 


26 


Nurse staffing and patient outcomes model 


[61] 


27 


A classification scheme for outcomes indicators 


[62] 


28 


Components of quality rating 


[63] 


29 


National voluntary consensus standards for nursing sensitive care 


[64] 


30 


CalNOC variables for the virtual dashboard 


[65] 


31 


Nursing sensitive indicators 


[66] 



reflected in heavy workload, unpredictable and inflexible 
schedules, excessive overtime, and threat of job loss, in- 
fluence workforce maintenance by limiting the capacity 
to recruit new nurses and retain those already employed. 
It has been demonstrated that the same factors are sig- 
nificant predictors of work-related illnesses, injuries and 
job dissatisfaction, which in turn lead to increased ab- 
senteeism and turnover [68-71]. The costs associated 
with these adverse effects come in diverse forms: prod- 
uctivity costs, wage replacements and disability payouts. 

Beyond the ability to manage the inevitable outflow of 
resources, a perennial challenge for every organization is 
to sustain its economic capacity to acquire and maintain 



the resources needed. This sustainability reflects a unit s 
or organizations ability to obtain needed inputs as eco- 
nomically as possible (that is, obtaining appropriate 
quality resources at least cost). Other indicators, such as 
cost per case-mix, highlight the issue of productivity and 
the imperative of maximizing the outputs produced 
from a given set of inputs or, alternatively, minimizing 
the inputs of nursing tasks, materials, and equipment to 
produce nursing services. 

Thus, with regard to this first function, nursing system 
performance covers a set of four dimensions that were 
operationalized into 14 variables. Five of the 31 models 
have no clear variables related to this subsystem. In 
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the remaining 26, the number of variables used to 
conceptualize this subsystem varies widely, from one to 
11. The most prevalent variables were related to nursing 
staff supply: quantity/intensity (17 times), quality (18) and 
skill mix (18). The issue of cost per case-mix figures in 11 
models. 

The usefulness of indicators related to this first subsys- 
tem for assessing nursing care performance may be justi- 
fied on several grounds: 

Their potential to detect latent failures at the "blunt end" of 
the nursing system 

The indicators mentioned above may reveal a broad set 
of dysfunctions and deficiencies, such as insufficient 
supply of basic material resources, gaps between nurse 
staffing and patients' needs, heavy workload and staff in- 
stability. Such deficiencies are considered as latent fail- 
ures that do not involve the practitioner and do not 
result from direct contact with the patients but that may 
have potentially delayed consequences on both nursing 
care processes and outcomes. As an example, inappro- 
priate nurse staffing may not necessary result in immedi- 
ate harm for patients but reveals a latent failure that 
may force nurses to reduce the surveillance function, 
thus creating unsafe delivery conditions that may ultim- 
ately result in adverse events among patients. 

Their sensitivity to nursing management 

These indicators' sensitivity to nursing management (e.g. 
staffing decisions) gives them an explanatory value for 
differing performances across care units or organiza- 
tions. The four dimensions that define this subsystem 
are all modifiable factors that result from managerial de- 
cisions and can be enhanced through policy initiatives at 
the unit, organizational and system levels. 

Their linkage with nursing care processes 

Assessing this function makes it possible to judge 
whether nursing care is being provided under conditions 
that are conducive to optimal nursing processes. Al- 
though the link between structural attributes and care 
processes has been shown to be inconsistent, the litera- 
ture reviewed contains a number of examples illustrating 
the potential application of structural indicators to 
predicting the performance of nursing processes. Re- 
search has shown that nursing inputs influence RNs' 
ability to care for patients, therefore affecting surveil- 
lance and other processes of care [72]. Higher patient- to 
-RN ratios and nursing workload have been associated 
with more tasks left undone by RNs and rationing of 
nursing care [49,73]. Low levels of staffing or heavy 
workload may result in reductions in time spent by 
nurses collaborating and communicating with other pro- 
viders and patients, thereby affecting the quality of both 



interprofessional coordination [74] and nurse-patient 
communication [75]. 

Their linkage with patient outcomes 

Measures of such structural aspects may be also useful 
to predict patient outcomes or to specif)^ the conditions 
under which outcomes are produced. A number of stud- 
ies and systematic reviews suggest that higher staffing 
levels and a richer staff-mix and skill-mix may be associ- 
ated with better outcomes and fewer adverse events for 
patients [76,77]. Conversely, poor working conditions, 
inadequate staffing resulting in heavy workload or ex- 
tended working hours, and problems reflective of staff 
maintenance, such as high turnover and excessive 
utilization of temporary staff from external agencies, 
have been linked with increasing risks to patient safety 
[78-81]. 

Transforming nursing resources into relevant nursing 
services 

The effective stewardship of nursing resources, as de- 
scribed above, is not an end state. A second key function 
of the nursing system is to transform available resources 
into nursing services that address patients' needs. This 
function involves a broad set of processes and mecha- 
nisms that reflect not only what staff nurses do for, with, 
and on behalf of patients and their families, but also 
what nursing managers do to support nurses' work and 
create an appropriate practice environment. It encom- 
passes, as well, how patients are engaged in their own 
care processes and how both staff and patients live their 
experience of care. Among the 31 models inventoried, 
28 include indicators related to this function. Those are 
covered in four interrelated dimensions, described 
below. 

The nursing processes 

Nursing care provision involves a range of individual, 
family, community and population-based approaches. It 
is materialized through interventions and processes that 
reflect the deployment of nurses' scope of practice, in- 
cluding assessment, planning and evaluation, problem 
and symptom management, health promotion and illness 
prevention, care coordination, and discharge planning. 
From a provider perspective, they capture the technical 
aspects of care and reflect the degree to which staff are 
able to use all their competencies and deploy their full 
scope of practice. Such processes also reflect the ability 
to provide care that meets the patient's overall needs. 

Nurses' practice environments 

While effective nursing interventions are considered an- 
tecedents of health outcomes, nurses' ability to perform 
these interventions is closely and consistently associated 
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with organizational processes that define the nursing 
practice environment and mediate the outcomes [54,82]. 
These processes have been conceptualized as interven- 
tions to support nursing work and create a professional 
environment for nurses [83]. Many of the indicators for 
capturing these processes that emerged from this review 
echo the characteristics of magnet organizations. These 
indicators include: support for nurses from co-workers 
and administration; communication and collaboration 
among nursing staff; collegiality between nurses and 
other professional groups; a motivating work climate; 
decentralization of decision-making with responsibilities 
for nursing services devolved to the nursing unit; and 
autonomy for nurses in their role. 

Patient experience and professional satisfaction 

These two dimensions are to a certain extent contingent 
upon the first two. Patient experience can be considered 
the result of clinical and organizational processes that 
should optimally ensure patients receive the right nurs- 
ing care, at the right time, and in the right way for them. 
Such a measure is essential to assess the acceptability 
and appropriateness of nursing care from the patient 
perspective. A number of frameworks have been devel- 
oped to conceptualize patient experience [20,84-87]. Yet 
the central theme, overarching all others in the nursing 
literature reviewed, relates to the patient-centredness of 
nursing care. Holistic, patient-centred care has often 
been seen as the essence of nursing care and interpreted 
as nurses' ability to communicate effectively with pa- 
tients, work with patients' expectations and values, en- 
gage patients in their care, deliver services that are 
sensitive to patients' needs, and ensure patients' smooth 
transition across the care system [88,89]. This notion 
covers not only the technical and organizational aspects 
of nursing processes, but also their interpersonal dimen- 
sion. It is reflected in the 31 models through a number 
of indicators that we grouped inductively into three at- 
tributes: 1) care continuity, which includes diverse 
facets, such as coordination of care, patient transition, 
and intra- and interprofessional collaboration; 2) pa- 
tient/family involvement, which is concerned with en- 
gaging patients and their families in their care, and the 
types of support provided to them to empower them 
and foster self-care; and 3) responsiveness, which also 
encompasses several facets, including respect for 
patients' preferences, quality of communication with 
patients and their families and degree of comprehensive- 
ness of care. 

Nurses' professional satisfaction may be also consid- 
ered the result of nursing processes. Although nurses 
can be rewarded through different facets of their work, 
their professional satisfaction mostly results from spe- 
cific aspects that influence their perception of their 



ability to accomplish their daily assignments and enjoy 
the work itself. As such, professional satisfaction relates 
to those Herzberg motivator factors and intrinsic re- 
wards that are directly linked with the work itself (e.g. 
responsibility, achievement, growth) as opposed to hy- 
giene factors (e.g. salary, working conditions, supervi- 
sion) [90]. Measures of professional satisfaction have 
also built upon the work of Hinshaw and colleagues, for 
whom nurses' job satisfaction is a combined function of 
their perception of the quality of care they provide, hav- 
ing the time to do their job, and the enjoyment derived 
from it [91]. 

Table 3 summarizes how this second subsystem is con- 
ceptualized and operationalized within the 31 conceptual 
models reviewed. In total, the review identified 10 vari- 
ables split across the four dimensions. Only three 
models have no indicator related to this subsystem. For 
the remaining 28, the number of variables used to 
conceptualize this subsystem ranges from one to seven. 
The most prevalent variables in these models are pa- 
tient/family involvement (13 times); problems and 
symptoms management (13), nursing work practice en- 
vironment characteristics (12), assessment, planning and 
evaluation (8) and care continuity (8). 

The usefulness of these four sets of indicators may be 
justified on three main grounds: 

Their potential to detect active or imminent failures at the 
"sharp end" of the nursing system 

An intrinsic advantage of indicators related to this func- 
tion is that they are more sensitive to differences in 
quality of care than are structure and outcome 
indicators. They may reveal dysfunctions and deficien- 
cies in the clinical microsystem, where providers interact 
directly with patients to deliver care. Inappropriate nurse 
interventions, miscommunication between nursing team 
members, and continuity gaps are examples of dysfunc- 
tions that create holes in the nursing system's front line 
of defense. Such holes are immediate causes of imminent 
or active failures that undermine the patient's experience 
of nursing care, compromise the provision of adequate 
services, predispose those services to errors, and ultim- 
ately may result in adverse outcomes. 

Their sensitivity to nurse interventions 

These indicators refer to the essence of what nurses do 
to improve patients' conditions [92]. They provide in- 
sights into the clinical and organizational processes that 
reflect both how nurses enact their roles and the condi- 
tions experienced by patients in their interactions with 
the nursing system. They also provide information that 
is actionable. Knowing which processes are ineffective is 
essential for both readjusting how nursing care is pro- 
vided and redeploying nursing resources. 
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Their linkage with patient outcomes 

Specific nursing processes, characterized by patient involve- 
ment, patient education, self-care assistance and psycho- 
logical support, have been associated with better outcomes 
for patients [93-95]. Doran and colleagues provided evidence 
that patients' therapeutic self-care ability was predicted by 
nurses' coordination of care, communication with other 
health care providers, and provision of high-quality technical 
care [38]. Other studies have shown that organizational 
models of nursing care which result in greater nurse auton- 
omy, more control by nurses of resources, and better rela- 
tions between nurses and physicians yield better patient 
satisfaction and outcomes [96,97]. 

Producing positive changes in patients' conditions 

Beyond the imperatives of effective stewardship of nursing 
resources and their transformation into services that meet 
high standards of quality, the desired end result of the inter- 
actions between patients, nursing staff, and nursing pro- 
cesses is to produce outcomes that lead to positive changes 
in a patients functional status, disease state or evolving con- 
dition. All 31 models include indicators related to this third 
function. Using an inductive process, we identified six dis- 
tinct but interrelated dimensions from the plethora of out- 
come indicators in the literature reviewed. The first five 
categories are considered nursing-sensitive outcomes, that 
is, outcomes that are affected, provided, and/or influenced 
by nursing personnel [80,98], although nursing may not be 
exclusively responsible for them. The sixth category includes 
outcomes to which nurses may contribute to some extent, 
but which are less informative for assessing nursing inter- 
ventions because these outcomes may be more influenced 
by other systems. 

Outcomes reflecting patient safety 

Nursing care services are provided to patients in an en- 
vironment with complex interactions that can generate 
harm, errors and unintended outcomes. Because of this, 
patient safety is largely considered an indicator of high- 
performing nursing care. This is related to nurses' roles 
in integrating care, detecting hazards and, ultimately, 
preventing errors and adverse events [99,100]. Failures 
in fulfilling these roles may result in errors of omission 
and commission, as well as adverse events. This review 
identified a number of safety- related outcomes consid- 
ered potentially sensitive to nursing: patient falls, injur- 
ies, medication errors, pulmonary infections, pressure 
ulcers, urinary tract infections, intravenous infections, 
abuses, and failure to rescue. 

Outcomes reflecting patient comfort and quality of life 
related to care 

Beyond harm prevention, the value of nursing care ser- 
vices lies also in their potential to relieve discomfort and 



improve the quality of life in the context of care. From 
this standpoint, nursing system performance reflects the 
extent to which patients' needs in relation to personal 
hygiene, nutrition, management of symptoms (pain, dys- 
pnea), and continence are met, unnecessary interven- 
tions (physical or chemical restraints, nasogastric tubes, 
prolonged use of urinary catheters) are avoided, and pa- 
tients' respect is ensured throughout the episode of care. 

Outcomes reflecting changes in patients' knowledge, skills, 
and behaviours 

Such outcomes reflect patients' ability to influence their 
own conditions through self-care and new behaviours. 
The indicators here focus on knowledge gained, attitudes 
changed, skills acquired, and behaviours adopted as a 
result of nursing interventions. Examples include pa- 
tients' knowledge of the prescribed treatment, ability to 
recognize symptoms, ability to carry out treatments as 
prescribed, and demonstration of health promoting 
behaviours. 

Outcomes reflecting patients' functional status 

This category of indicators covers essential end results 
and benefits that reflect what happens in people's lives 
as a result of nursing care interventions. In the models 
examined, these indicators encompass diverse aspects of 
patients' general functional status and conditions, in- 
cluding physical, psychosocial and cognitive status. 

Outcomes reflecting patient satisfaction with the care 
experience 

These indicators focus on patients' satisfaction with their 
care experience and the evolution of their condition. 
This subjective evaluation by patients reflects the inter- 
action of their expectations of care and their perceptions 
of actual outcomes resulting from provider services 
[101]. The nature of nurses' responsibilities is such that 
they are potentially well positioned to influence patients' 
conditions and how the health system functions across 
all aspects of patient care. Hence, patients' satisfaction 
with nursing care is considered in several models to be 
both an outcome of nursing services and a primary de- 
terminant of overall satisfaction with an episode of care. 

Outcomes reflecting a joint contribution of nursing care 
and other care systems 

Nursing as a "whole system" interacts with several other 
"whole systems" to achieve outcomes. Positive change in 
patients' health status is an ultimate outcome that most 
often reflects the contribution of several systems of care 
and many healthcare disciplines. Conversely, negative 
outcomes such as mortality, morbidity, complications, 
extended length of stay and readmissions reflect failures 
that may have multiple causes and can originate from 
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one or more systems. Although such events cannot eas- 
ily be imputed either to nursing or to any specific care 
system, they still reflect outcomes of each of these sys- 
tems and warrant consideration in assessing the per- 
formance of any system. 

Table 4 shows how this third function is conceptual- 
ized and operationalized in the models reviewed. In 
total, the review identified 27 variables, which we 
grouped into six dimensions. All 31 models include out- 
come indicators. The number of variables used to 
conceptualize this subsystem ranges from one to 15. The 
most prevalent variables cut across the different categor- 
ies of nursing-sensitive outcomes and include symptom 
management (22 times), skin integrity (17), falls (17), pa- 
tient satisfaction (17), physical functional capacity (17), 
cognitive and psychosocial functional capacity (15), 
medication management (14), nosocomial infections (12) 
and ability to achieve appropriate self-care (11). The 
linkages between these outcomes related indicators and 
the two other subsystems have been described in the 
previous sections. 

Conclusions 

Across the three subsystems, at least 51 variables, 
grouped into 14 dimensions, have been used to assess 
these subsystems' performance in achieving their func- 
tions. Cross-validation by examination of the 70 add- 
itional papers revealed no additional generic variables, 
but in some cases, specific indicators related to a specific 
clinical domain and linkages between those variables. 
Figure 3 summarizes the dimensions and indicators in- 
ventoried for the three subsystems, based on the review. 

While all 51 variables are valid indicators that together 
represent the entire domain of nursing care performance 
as a construct, it can be seen in Tables 2, 3 and 4 that 
no individual model covers this entire domain. Rather, 
the different models show a variety of combinations of 
potential variables, reflecting the diversity of options for 
measuring nursing care. However, it is worth noting 
that, for nearly all the models, the scope of nursing care 
performance variables extends to more than one subsys- 
tem. In fact, 25 of the 31 models include indicators re- 
lated to all three subsystems. Of the others, three focus 
on the second and third subsystems, one includes indi- 
cators related to the first and third subsystems, and two 
restrict their scope to outcome indicators. 

From a policy perspective, this review clearly shows 
that a systems approach provides a useful and powerful 
tool to build a comprehensive framework for perform- 
ance evaluation of both the overall nursing system and 
its subsystems. By incorporating the three functions 
described above, the NCPF offers an effective way to 
compile a complete picture of overall nursing care per- 
formance in any setting. It suggests a shift away from a 



narrow view of nursing performance to a broader per- 
spective that encompasses more complex and more 
complete measures that take into account extended as- 
pects of nursing activities. As such, the NCPF s systemic 
perspective may help avoid many of the dysfunctional 
strategies that often guide performance measurement 
[102-105]. One such dysfunctional strategy, often de- 
scribed as tunnel vision, consists of concentrating on 
certain limited areas of performance while neglecting 
other important but unmeasured areas. Another such 
strategy is to focus on short-term issues and proximal 
outcomes that are more easily detectable, to the exclu- 
sion of long-term issues or distal outcomes, which may 
be more difficult to measure or take some time to be- 
come measurable. A third strategy is to focus on indica- 
tors that are more readily available (in most cases, 
structural indicators) while overlooking more complex 
measures. Finally, another strategy is to focus on isolated 
indicators without taking into account the many interac- 
tions between the different components of the system. 
The NCPF may help mitigate these dysfunctions. This 
framework offers managers appropriate guidance to de- 
vise a set of performance indicators that encompasses all 
functional areas of the nursing system, covers its struc- 
tural features as well as processes and outcomes, con- 
siders both short-term and long-term issues, and 
provides a cross -functional view of the nursing system 
and of the interdependence among its different 
functions. While many aspects of nursing care may re- 
main irreducibly invisible to policy-makers, the NCPF 
brings attention to a wide array of definable and measur- 
able dimensions that can be used to assess nursing 
contribution. 

However, while such a framework reflects the multidi- 
mensionality of nursing care performance, it poses the 
serious challenge of selecting a core set of indicators 
from among multiple options. The challenge is to find a 
fair balance between a narrow, easily implemented set of 
indicators that provides a limited picture of performance 
and a larger, more complex, but unwieldy set of indica- 
tors that gives a more complete picture. Some authors 
warn that having an unbalanced and overabundant set of 
performance measures can be counterproductive. Push- 
ing staff to deliver an unreasonable number of targets 
may increase stress, lower morale and ultimately com- 
promise quality and performance [105]. The NCPF may 
be a useful tool to address such challenges. 

However this study only achieves the first requirement 
in the development of any performance measurement 
system, which is to develop a robust conceptual frame- 
work within which performance indicators can be identi- 
fied and developed. The foundations provided by the 
NCPF would ensure that all major areas of nursing care 
performance are covered by any measurement system. 
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Nursing care environment: legislation, regulation, professional systems, social and economic issues, culture, 

educational systems 
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Figure 3 The nursing care performance framework. 



that priorities for developing new measures or instru- 
ments can be identified, and that data collection efforts 
on performance are not misdirected or duplicated. 

While the NCPF provides a matrix of indicators that 
covers the key functions of the nursing system, this 
framework is based on the principle that performance 
measurement, by its dynamic nature, must address both 
short-term and long-term issues and be constantly ad- 
justed to the challenges of particular domains at particu- 
lar times [106-108]. While the framework specifies what 



core aspects should be measured to assess nursing care 
performance, it allows for variations in the combinations 
of indicators used to measure those aspects in different 
contexts of care. Many jurisdictions are faced with the 
challenge of developing a realistic set of performance in- 
dicators with sufficient breadth and depth to capture the 
spectrum of nursing care. The NCPF provides a com- 
mon ground from which to define performance, decide 
what is meaningful and important with regard to each 
nursing subsystem or function, devise a common and 
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balanced set of performance indicators for a given sector 
of nursing care, and derive benchmarks for this sector. 
In short, the challenge for policy-makers is to select 
from among the universe of variables incorporated into 
this framework to develop, for the different sectors of 
nursing, an optimal and common portfolio of perform- 
ance indicators that do not simply focus on measuring 
what is available or easy to measure, but are consistent 
with their objectives, account for their context of care 
and are relevant to the needs of the key stakeholders. 
Such selection must be informed by explicit criteria for 
ensuring that performance indicators exhibit the key 
characteristics of acceptability, feasibility, reliability, sen- 
sitivity to change and validity [109-111]. 

Developing a performance measurement system for 
nursing care is also about collecting, computing and pre- 
senting performance data for the purposes of following 
up, monitoring and improving organizational and clin- 
ical performance. Many of the current efforts to use per- 
formance data tend to concentrate on collecting and 
organizing existing administrative information and dis- 
seminating it for limited management applications [112]. 
To use this framework to best advantage, policy-makers 
and managers must invest in more creative ways of 
collecting, analyzing and reporting the data, and ultim- 
ately applying that information to improve nursing per- 
formance. The diverse nature of the dimensions of 
nursing care performance, the multiple possible uses of 
the performance data, and the variety of audiences in- 
volved may necessitate a wide variety of data collection 
methods, analytical techniques and dissemination strat- 
egies to assemble this information, make it accessible to 
different groups, and ensure that positive action results 
from it. 

Despite the contributions mentioned above, some lim- 
itations remain in this study. Although we used a highly 
elaborated and systematic protocol to conduct the re- 
view, the approach was challenged by the broad nature 
of the topic being reviewed. It required a breadth of 
search and a labour-intensive process that allowed us to 
identify a rich set of performance-related variables and 
linkages between them. However, the number of themes 
and issues to cover may have limited the depth of the re- 
view on specific aspects of performance measurement in 
nursing care. More in-depth complementary reviews will 
be needed to document specific dimensions or 
subdimensions of this framework or to compile more 
thorough evidence on the linkages identified between 
those dimensions. While the review revealed some indi- 
cators that are more prevalent in existing conceptual 
models or are supported by strong research evidence, it 
is premature to conclude those elements are more sig- 
nificant than others that have been the subject of less at- 
tention or research. This is an unfinished agenda that 



opens the door for extensive research programs. Further 
studies will be needed to assess the implementation of 
the NCPF in different contexts of nursing care, gain fur- 
ther insight into the linkages hypothesized in the frame- 
work, and compile more evidence on the indicators that 
are more sensitive to nursing and that address the needs 
of the key stakeholders involved in care. Finally, another 
limitation may be that, while our search focused on the 
performance literature, we may have overlooked some 
key dimensions that do not relate directly to perform- 
ance but may be essential to ensure a systems 
performance. 

Competing interests 

The authors declare they have no competing interests. 
Authors' contributions 

The study was conceived and designed by Carl-Ardy Dubois and Danielle 
D'Amour. All authors made a substantive contribution to the four phases of 
this systematic review, as described in the methods section. Carl-Ardy 
Dubois prepared the first draft of this manuscript. All authors contributed to 
revising the manuscript. All authors read and approved the final manuscript. 

Acknowledgements 

This study was funded by the Canadian Institutes of Health Research. 
Author details 

^Faculty of Nursing, University of Montreal, Montreal, Canada. ^Department 
of Health Administration, Faculty of Medicine, University of Montreal, 
Montreal, Canada. 

Received: 15 July 2012 Accepted: 7 February 2013 
Published: 7 March 2013 

References 

1. Jelinek R, Pierce F: A nursing systems approach: productivity and quality 

of care. Public Prod Rev 1982, 6(3):223-240. 

2. Pappas SH: The cost of nurse-sensitive adverse events, J Nurs Adm 2008, 
38(5):230-236. 

3. McCloskey JC: Help to make nursing visible, J Nurs Scholorsh 1 995, 

3:170-175. 

4. Lang NM, Mitchell PH, Hinshaw AS, Jennings BM, Lamb GS, Mark BA, Moritz 
P: Foreword: measuring and improving healthcare quality, Med Care 

2004, 42(2):1-3. 

5. Maas ML, Delaney C: Nursing process outcome linkage research: issues, 
current status, and health policy implications, Med Care 2004, 
42(2):II40-II48. 

6. Kennedy MA, Hannah K: Representing nursing practice: evaluating the 
effectiveness of a nursing classification system. Can J Nurs Res 2007, 
39(1):58-79. 

7. Zielstorff RD: Capturing and using clinical outcome data: implications for 
information systems design. J Am Med Inform Assoc 1995, 2(3):191-196. 

8. Hajewski C, Maupin JM, Rapp DA, Sitterding M, Pappas J: Implementation 
and evaluation of Nursing Interventions Classification and Nursing 
Outcomes Classification in a patient education plan, J Nurs Care Qual 
1998, 12(5):30-40. 

9. Aiken LH, Clarke SP, Sloane DM: Hospital restructuring: does it adversely 
affect care and outcomes? J Nurs Adm 2000, 30(1 0):45 7-465. 

10. Gregg AC: Performance management data systems for nursing service 
organizations, J Nurs Adm 2002, 32(2):71-78. 

1 1 . Behrenbeck JG, Timm JA, Griebenow LK, Demmer KA: Nursing-sensitive 
outcome reliability testing in a tertiary care setting, Int J Nurs Terminol 
Classif 2005, 1 6(1 ):1 4-20. 

12. Riehle Al, Hanold LS, Sprenger SL, Loeb JM: Specifying and standardizing 
performance measures for use at a national level: implications for 
nursing-sensitive care performance measures, Med Care Res Rev 2007, 
64(Suppl 2):64S-81S. 



Dubois et al. BMC Nursing 2013, 12:7 
http://www.bionnedcentral.conn/1 472-6955/1 2/7 



13. Doran D, Mildon B, Clarke S: Toward a National Report Card in Nursing: A 
Knowledge Synthesis. Toronto: Nursing Health Services Research Unit and 
University of Toronto; 201 1. 

14. Kurtzrman ET, Kizer KW: Evaluating the performance and contribution of 
nurses to achieve an environment of safety. Nurs Adm Q 2005, 
29(1):14-23. 

15. Harris M, Vanderboom C, Hughes R: Nursing-sensitive safety and quality 
outcomes: the taming of a wicked problem. AppI Nurs Res 2009, 
22(2):146-151. 

16. Adair CE, Simpson E, Casebeer AL, Birdsell JM, Hayden KA, Lewis S: 
Performance measurement in healthcare: part l-concepts and trends 
from a state of the science review. Healthc Policy 2006, 1(4):85-104. 

17. Needleman J, Kurtzman ET, Kizer KW: Performance measurement of 
nursing care: state of the science and the current consensus. Med Care 
Res Rev 2007, 64(Suppl 2):10S-43S. 

18. Idvall E, Rooke L, Hamrin E: Quality indicators in clinical nursing: a review 
of the literature. J Nurs 1997, 25(1):6-17. 

19. Arah OA, Klazinga NS, Delnoij DM, ten Asbroek AH, Custers T: Conceptual 
frameworks for health systems performance: a quest for effectiveness, 
quality, and improvement. Int J Qual Health Care 2003, 15(5):377-398. 

20. Institute of Medicine: Crossing the Quality Chasm: A New Health System for 
the Twenty-first Century Washington, DC: National Academies Press; 2001. 

21. Greenhaigh T, Robert G, MacFarlane F, Bate P, Kyriakidou 0: Diffusion of 
innovations in service organizations: systematic review and 
recommendations. Milbank Q 2004, 82(4):581-629. 

22. Lavis J, Davies H, Oxman A, Denis JL, Golden-Biddle K, Ferlie E: Towards 
systematic reviews that inform health care management and policy- 
making. J Health Serv Res Policy 2005, 10(Suppl l):35-48. 

23. Tranfield D, Denyer D, Smart P: Towards a methodology for developing 
evidence-informed management knowledge by means of systematic 
review. Brit J Manage 2003, 14(3):207-222. 

24. Mays N, Pope C, Popay J: Systematically reviewing qualitative and 
quantitative evidence to inform management and policy-making in the 
health field. J Health Serv Res Policy 2005, 10(Supp l):6-20. 

25. Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A, Harvey J, Hsu 
R, Katbamna S, Olsen R, Smith L, Riley R, Sutton AJ: Conducting a critical 
interpretive synthesis of the literature on access to healthcare by 
vulnerable groups. BMC Med Res Methodol 2006, 6:35. 

26. Dubois C-A, Singh D: From staff-mix to skill-mix and beyond: towards a 
systemic approach to health workforce management. Hum Resour Health 
2009, 7:87. 

27. Bowker GC, Star SL, Spasser MA: Classifying nursing work. Online J Issues 
Nurs 200], 6:2. 

28. Canadian Nurses Association: International Classification for Nursing 
Practice: documenting nursing care and client outcomes. Nursing Now 

2003, 14:1-4. 

29. Hughes RG: Patient Safety and Quality: An Evidence-based Handbook for 
Nurses. Agency for Healthcare Research and Quality: Rockville, MD; 2008. 

30. Savitz LA, Jones CB, Bernard S: Quality indicators sensitive to nurse 
staffing in acute care settings. In Advances in Patient Safety: From 
Research to Implementation. Edited by Henriksen K, Battles JB, Marks 
ES, Lewin D. Rockville, MD: Agency for Healthcare Research and 
Quality; 2005:375-385. Volume 4: Programs, Tools, and Products. 

31 . Koch T: A review of nursing quality assurance. J Adv Nurs 1 992, 
17(7):785-794. 

32. Henriksen K, Dayton E, Keyes MA, Carayon P, Hughes R: 
Understanding adverse events: a human factors framework. In 
Patient Safety and Quality: An Evidence-based Handbook for Nurses. 
Edited by Hughes RG. Rockville, MD: Agency for Healthcare Research 
and Quality; 2008:1-67-1-85. 

33. Atchley SJ: A time-ordered, systems approach to quality assurance 
in long-term care. J AppI Gerontol 1991, 1 0(1 ):1 9-34. 

34. Donabedian A: Models for organizing the delivery of health services and 
criteria for evaluating them. Milbank Q 1972, 50(4):103-154. 

35. Parsons T: Structure and Process in Modern Societies. New York: Free Press; 
1960. 

36. Sicotte C, Champagne F, Contandriopoulos A-P, Barnsley J, Beland F, 
Leggat SG, Denis J-L, Bilodeau H, Langley A, Bremond M, Baker GR: 
A conceptual framework for the analysis of health care 
organizations' performance. Health Serv Manage Res 1998, 
1(1):24-41. discussion 41-48. 



Page 1 9 of 20 



37. Reid PP, Compton WD, Grossman JH, Fanjiang G: Building a Better Delivery 
System: A New Engineering/Health Care Partnership. Washington, DC: National 
Academies Press; 2005. 

38. Doran Dl, Sidani S, Keatings M, Doidge D: An empirical test of the 
NursingRole Effectiveness Model J Adv Nurs 2002, 38(l):29-39. 

39. Irvine D, Sidani S, Hall LM: Finding value in nursing care: a framework for 
quality improvement and clinical evaluation. Nurs Econ 1998, 

16(3):1 10-1 16,131. 

40. Wong ST, Stewart AL, Gilliss CL: Evaluating advanced practice nursing care 
through use of a heuristic framework. J Nurs Care Qual 2000, 14(2):21-32. 

41. Gallagher RM, Rowell PA: Claiming the future of nursing through nursing- 
sensitive quality indicators. Nurs Adm Q 2003, 27(4):273-284. 

42. Sidani S, Irvine D: A conceptual framework for evaluating the nurse 
practitioner role in acute care settings. J Nurs 1999, 30(l):58-66. 

43. Pierce SF: Nurse-sensitive health care outcomes in acute care settings: an 
integrative analysis of the literature. J Nurs Care Qual 1997, 1 1(4):60-72. 

44. Chang BL, Lee JL, Pearson ML, Kahn KL, Elliott MN, Rubenstein LL: 
Evaluating quality of nursing care: the gap between theory and practice. 
J Nurs Adm 2002, 32(7-8):405-418. 

45. Unruh L, Wan TT: A systems framework for evaluating nursing care 
quality in nursing homes. J Med Syst 2004, 28(2):1 97-214. 

46. Gerolamo AM: The conceptualization of physical restraint as a nursing- 
sensitive adverse outcome in acute care psychiatric treatment settings. 
Arch Psychiatr Nurs 2006, 20(4):175-185. 

47. Attree M: An analysis of the concept "quality" as it relates to 
contemporary nursing care. Int J Nurs Stud 1993, 30(4):355-369. 

48. Stetler CB, Morsi D, Burns M: Physical and emotional patient safety. A 
different look at nursing-sensitive outcomes. Outcomes Manag Nurs Pract 
2000, 4(4):159-165. quiz 165-166. 

49. Schubert M, Glass TR, Clarke SP, Aiken LH, Schaffert-Witvliet B, Sloane DM, 
De Geest S: Rationing of nursing care and its relationship to patient 
outcomes: the Swiss extension of the International Hospital Outcomes 
Study. Int J Qual Health Care 2008, 20(4):227-237. 

50. Rice SM, Van Slobbe A, Rathgeber D: Nursing practice evaluation using an 
expert panel process. Clin Govern Int J 2007, 12(2):93-101. 

51. McGillis Hall L: Report cards: relevance for nursing and patient care 
safety. Int Nurs Rev 2002, 49(3):168-177. 

52. McGillis Hall L, Doran D, Laschinger HS, Mallette C, Pedersen C, O'Brien- 
Pallas LL: A balanced scorecard approach for nursing report card 
development. Outcomes Manag 2003, 7(1 ):1 7-22. 

53. Maas ML, Reed D, Reeder KM, Kerr P, Specht J, Johnson M, Moorhead S: 
Nursing outcomes classification: a preliminary report of field testing. 
Outcomes Manag 2002, 6(3):1 12-119. 

54. Sidani S, Doran DM, Mitchell PH: A theory-driven approach to evaluating 
quality of nursing care. J Nurs Scholarsh 2004, 36(l):60-65. 

55. Doran DM, Harrison MB, Laschinger HS, Hirdes JP, Rukholm E, Sidani S, Hall 
LM, Tourangeau AE: Nursing-sensitive outcomes data collection in acute 
care and long-term-care settings. Nurs Res. 2006, 55(2 Suppl):S75-81. 

56. Given BA, Sherwood PR: Nursing sensitive patient outcomes-a white 
paper. Oncol Nurs Forum 2005, 32(4):773-84. 

57. Maas ML, Johnson M, Moorhead S: Classifying nursing-sensitive patient 
outcomes. Image J Nurs Sch 1996, 28(4):295-301. 

58. Carayon P, Gurses AP: A human factors engineering conceptual 
framework of nursing workload and patient safety in intensive care 
units. Intensive Crit Care Nurs 2005, 21(5):284-301. 

59. Lancaster DR, King A: The spider diagram nursing quality report card: 
bringing all the pieces together. J Nurs Adm, 29(7-8):43-48. 

60. Jennings BM, Loan LA, DePaul D, Brosch LR, Hildreth P: Lessons learned 
while collecting ANA indicator data. J Nurs Adm 2001, 31(3):121-9. 

61. Cho SH: Nurse staffing and adverse patient outcomes: a systems 
approach. Nurs Outlook 2001, 49(2):78-85. 

62. Jennings BM, Staggers N, Brosch LR: A classification scheme for outcome 
indicators. Image J Nurs Sch 1 999, 31 (4):381 -388. 

63. Pearson ML, Lee JL, Chang BL, Elliott M, Kahn KL, Rubenstein LV: Structured 
implicit review: a new method for monitoring nursing care quality. 
Med Care 2000, 38(1 1):1 074-1 091. 

64. NQF (National Quality Forum): National voluntary consensus standards for 
nursing-sensitive care: an initial performance measure set 
Washington, DC; 2004. 

65. Aydin CE, Bolton LB, Donaldson N, Brown DS, Mukerji A: Beyond nursing 
quality measurement: the nation's first regional nursing virtual 



Dubois et al. BMC Nursing 2013, 12:7 
http://www.bionnedcentral.conn/1 472-6955/1 2/7 



Page 20 of 20 



dashboard. In Advances in Patient Safety: New Directions and Alternative 
Approaches. Volume 1: Assessment Edited by Henriksen K, Battles JB, Keyes 
MA, Grady ML Rockville, MD: Agency for Healthcare Research and Quality; 
2008:217-234. 

66. ANA (American Nurses Association): Nursing Sensitive Indicators. 201 1. http:// 
www.nursingworld.org/MainMenuCategories/ 
ThePracticeofProfessionalNursing/PatientSafetyQuality/Research- 
Measurement^he-National-Database/Nursing-Sensitive-lndicators_1. Last 
accessed: March 14, 2013. 

67. Cowing M, Davino-Ramaya C, Ramaya K, Szmerekovsky J: Health care 
delivery performance: service, outcomes, and resource stewardship. 
Perm J 2009, 13:72-78. 

68. Kirkcaldy BD, Trimpop R, Cooper CL: Working hours, job stress, work 
satisfaction and accident rates among medical practitioners and allied 
personnel. Int J Stress Manag 1997, 4(2):79-87. 

69. Healy C, McKay M: Identifying sources of stress and job satisfaction in the 
nursing environment. Aust J Adv Nurs 1999, 17(2):30-35. 

70. Hoffman AJ, Scott LD: Role stress and career satisfaction among 
registered nurses by work shift patterns. J Nurs Adm 2003, 33(6):337-342. 

71. Hayes LJ, O'Brien-Pallas L, Duffield C, Shamian J, Buchan J, Hughes F, 
Spence Laschinger HK, North N, Stone PW: Nurse turnover: a literature 
review. Int J Nurs Stud 2006, 43:237-263. 

72. Aiken LH, Clarke SP, Sloane DM: Hospital staffing, organization, and 
quality of care: cross-national findings. Int J Qual Health Care 2002, 
14(1):5-13. 

73. Thomas-Hawkins C, Flynn L, Clarke SP: Relationships between registered 
nurse staffing, processes of nursing care, and nurse- re ported patient 
outcomes in chronic hemodialysis units. Nephrol Nurs J 2008, 
35(2):123-130. 145; quiz 131. 

74. Baggs JG, Schmitt MH, Mushlin Al, Mitchell PH, Eldredge DH, Oakes D, 
Hutson AD: Association between nurse-physician collaboration and 
patient outcomes in three intensive care units. Crit Care Med 1999, 
27:1991-1998. 

75. Davis S, Kristjanson LJ, Blight J: Communicating with families of patients 
in an acute hospital with advanced cancer: problems and strategies 
identified by nurses. Cancer Nurs 2003, 26:337-345. 

76. Currie V, Harvey G, West E, McKenna H, Keeney S: Relationship between 
quality of care, staffing levels, skill mix and nurse autonomy: literature 
review. J Adv Nurs 2005, 51 (1):73-82. 

77. Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K: Nurse-staffing 
levels and the quality of care in hospitals. N Engl J Med 2002, 
346(22):1715-1722. 

78. Rogers AE, Hwang WT, Scott LD, Aiken LH, Dinges DF: The working hours 
of hospital staff nurses and patient safety. Health Aff 2004, 23(4):202-212. 

79. Hickam DH, Severance S, Feldstein A, Ray L, Gorman P, Schuldheis S, Hersh 
WR, Krages KP, Helfand M: The Effect of Health Care Working Conditions on 
Patient Safety. Evidence Reports/Technology Assessments, Number 74. Rockville, 
MD: Agency for Healthcare Research and Quality; 2003. 

80. Institute of Medicine: Keeping Patients Safe: Transforming the Work 
Environments of Nurses. Washington, DC: National Academies Press; 2004. 

81. Prot S, Fontan JE, Alberti C, Bourdon 0, Farnoux C, Macher MA, Foureau A, 
Faye A, Beaufils F, Gottot S, Brion F: Drug administration errors and their 
determinants in pediatric in-patients. Int J Qual Health Care 2005, 
17:381-389. 

82. Aiken LH, Patrician PA: Measuring organizational traits of hospitals: the 
Revised Nursing Work Index. Nurs Res 2000, 49:146-153. 

83. Lake ET: Development of the practice environment scale of the Nursing 
Work Index. Res Nurs Health 2002, 25 (3): 1 76-188. 

84. Gerteis M, Edgman-Levitan S, Daley J, Delbanco T: Through the Patient's Eyes: 
Understanding and Promoting Patient-centered Care. San Fransisco: Jossey- 
Bass; 1993. 

85. Cronin C: Patient-centered Care: An Overview of Definitions and Concepts. 
Washington, DC: National Health Council; 2004. 

86. Johnson CL, Cooper PK: Patient-focused care: what is it? Holist Nurs Pract 
1997, 11(3):1-7. 

87. Sizmur S, Redding D: Key Domains of the Experience of Hospital Outpatients. 
Oxford, UK: Picker Institute Europe; 2010. 

88. McCormack B, McCance TV: Development of a framework for person- 
centred nursing. J Adv Nurs 2006, 56(5):472-479. 

89. McCance TV: Caring in nursing practice: the development of a 
conceptual framework. Res Theory Nurs Pract 2003, 17(2):101-1 16. 



90. Herzberg F: One more time: how do you motivate employees? Harv Bus 
Rev]987, 65(5):109-120. 

91. Hinshaw AS, Smeltzer CH, Atwood JR: Innovative retention strategies for 
nursing staff. J Nurs Adm 1987, 17(6):8-14. 

92. Lohr KN: Health outcomes methodology symposium. Summary and 
recommendations. Med Care 2000, 38(ll-9):1 94-208. 

93. Brooker C, Repper JM, Booth A: The effectiveness of community mental 
health nursing: a review. J Clin Effect 1996, 1(2):44-49. 

94. Singh D: Transforming Chronic Care: Evidence about Improving Care for 
People with Long-term Conditions. Birmingham, UK: University of 
Birmingham Health Services Management Centre; 2005. 

95. Poole K: The evolving role of the clinical nurse specialist within the 
comprehensive breast cancer care centre. J Clin Nurs 1996, 5:341-349. 

96. Aiken LH, Sochalski J, Lake ET: Studying outcomes of organizational 
change in health services. Med Care 1997, 35(1 1):NS6-18. 

97. Kutney-Lee A, McHugh MD, Sloane DM, Cimiotti JP, Flynn L, Neff DF, Aiken 
LH: Nursing: a key to patient satisfaction. Health Aff 2009, 28(4):669-677. 

98. National Quality Forum: National Voluntary Consensus Standards for Nursing- 
Sensitive Care: An Initial Performance Measure Set Washington, DC: National 
Quality Forum; 2004. 

99. Mitchell PH, Lang NM: Nurse staffing: a structural proxy for hospital 
quality? Med Care 2004, 42(1 ):1 -3. 

100. Rogers AE, Dean GE, Hwang WT, Scott LD: Role of registered nurses in 
error prevention, discovery and correction. Qual Saf Health Care 2008, 
17(2):117-121. 

101. Eriksen LR: Patient satisfaction with nursing care. J Nurs Meas 1995, 
3(1 ):5 9-76. 

102. Bouckaert G, Balk W: Public productivity measurement: diseases and 
cures. Public Prod Manage Rev 1991, 15(2):229-235. 

103. Smith P: On the unintended consequences of publishing performance 
data in the public sector. Int J Public Admin 1995, 18:277-310. 

104. Mannion R, Braithwaite J: Unintended consequences of performance 
measurement in healthcare: 20 salutary lessons from the English 
National Health Service. Intern Med J 2012, 42(5):569-574. 

105. Goddard M, Smith PC: Performance measurement in the new NHS. 
Health Policy Matters 2001, 3:1-8. 

106. Brook RH, McGlynn EA, Shekelle PG: Defining and measuring quality of 
care: a perspective from US researchers. Int J Qual Health Care 2000, 
12(4):281-295. 

107. Mant J: Process versus outcome indicators in the assessment of quality 
of health care. Int J Qual Health Care 2001, 13(6):475-480. 

108. Mainz J: Defining and classifying clinical indicators for quality 
improvement. Int J Qual Health Care 2003, 15(6):523-530. 

109. Hurtado MP, Swift EK, Corrigan JM: Envisioning the National Health Care 
Quality Report Washington, DC: National Academies Press; 2001. 

110. Kelley E, Hurst J: Health Care Quality Indicators Project: Conceptual Framework 
Paper OECD Health Working Papers, no.23. Paris: OECD; 2006. 

111. Mueller C, Karon SL: ANA nurse sensitive quality indicators for long-term 
care facilities. J Nurs Care Qual 2004, 19(1):39-47. 

112. Smith P, Mossialos E, Papanicolas I, Leatherman SL: Performance 
Measurement for Health System Improvement: Experiences, Challenges and 
Prospects. 2010th edition. Cambridge: Cambridge University Press; 2010. 



doi:1 0.1 186/1472-6955-12-7 

Cite this article as: Dubois et al.: Conceptualizing performance of 
nursing care as a prerequisite for better measurement: a systematic and 
interpretive review. BMC Nursing 2013 12:7. 



